Learning pullback HMM distances for action recognition

In action, activity or identity recognition it is sometimes useful, rather than to simply extract spatiotemporal features from the volumes representing motions, to explicitly encode their dynamics by means of dynamical systems. Hidden Markov models (HMMs) are a popular choice in that respect: actions can be then classified by measuring distances in the space of HMMs. However, using a fixed, arbitrary distance to classify dynamical models does not necessarily produce good classification results. In this paper we outline a framework based on pullback metrics which allows instead, given a training set of models of whatever class, to learn an optimal pullback distance tuned for that specific training set, starting from any base metric/distance/divergence. In particular we conduct here this analysis for HMMs, and show results on the KTH and Weizmann data-set illustrating the gain in classification performance this method delivers.